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SHAREM 


3  7  Years  of  Lessons  Learned 


Surface  Warfare  Center  of  Excellence 


□  Background/Overview 

□  Improved-SHAREM  Process 

□  Metrics 

□  Lessons  Learned 


Naval  Warfare  Center  of  Excellence 


Surface  Warfare  Center  of  Excellence 


>  Develops,  improves,  tests,  validates,  and  evaluates 
fleet  doctrine  and  platform-  and  mission-specific 
TTP 


>  Secondary  roles  in  enhancing  the  capability  of  the 
Navy  forces  to  operate  effectively  in  unilateral.  Joint, 
and  coalition  operations 

>  Additional  Duties  include: 

Responsibilities  In  the  areas  of  operational  test  and  evaluation, 
Research  and  development, 
v'  Experimentation, 

V  Modeling  and  simulation, 

V  War  gaming, 

v'  Requirements  definition, 

Joint  and  allied  doctrine  and  TTP  development. 


SWDG-21 


Surface  Warfare  Center  of  Excellence 


The  Readiness  Effectiveness  Measuring  (REM)  Program(s) 
Technical ...  Training ...  Tactical ... 


HowwbII  Itwork? 


Systems 
Sensors 
Weapons 

Command  and  Control 
Systems  Integration 


Dow^kmmhowlto 
operate  if  ? 


Systems 
Sensors 
Weapons 

Command  and  Control 
Systems  Integration 


Empiricai  Data 


Howisitmploy^d 
9g9inst  the  fitreef  ? 


Tactics,  Techniques  & 
Procedures 


gj  High  Level  Assessment  vs.  REM  , 

^  Surface  Warfare  Center  of  Excellence  - ^ 


>  High  Level  Assessments  conducted  at  the  integrated  or  Warfare 
Commander  &  above  level 


□  Primarily  ascertains  Staff’s  ability  to  effectively  plan  &  implement  a 
campaign  at  the  operational  &  tactical  level 

□  Planning  considerations  are  driven  by  expected  unit  performance 

□  Asset  allocation  driven  be  performance  trade-offs 

>  REM  analysis  conducted  at  the  unit  level  down  to  the  individual 
unit,  sensor  package  &  weapon 

□  Empirical  data  collected  provides  critical  performance  feedback  to  tactical 
&  operational  planners 

□  Allows  individual  units  to  ascertain  innate  capabilities  and  adjust 
operational  plans  at  the  tactical  level 

□  Identifies  performance  levels  and  sets  operational  capabilities 

□  Provides  optimal  asset  mix  for  various  tactical  considerations 

□  Identifies  potential  game  changer  systems  or  weapons 

□  Identifies  and  provides  capability  gap  non-materiel  solutions 
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REM  Program 


Surface  Warfare  Center  of  Excellence 


Where  is  the  Focus? 


Fundamentally  unit  performance 
determines  IPCLs  &  Gaps 


>  REM  programs  analyze  unit  performance 

□  Sensor  performance 

□  Weapon  effectiveness 

□  Human  factors 


□  Overall  system  effectiveness 

□  Causality  factors 

>  Integrated  assessment  is  the  Monday  morning 
Coaches  debate 


□  What  could  have  been  done  differently? 

□  How  well  did  we  execute  the  plan? 

>  Individual  units  make  the  plays  on  the  battlefield  -  not 
the  staff 


□  Unit  performance  determines  where  gaps  exist 

□  Gaps  determine  integrated  priority  capability  lists 


Find  -  Fix  -  Track  -  Target  -  Engage  -  Assess 

All  Unit  Level  Functions  -  Regardless  of  Warfare  Area 


Surface  Warfare  Center  of  Excellence 


What  does  it  Measure? 


REM  Program 


>  Find  -  sensor  capabilities  against  known  targets 

□  Structured  events  establish  optimal  performance  -  “as  good  as  it  gets”- 
involves  ability  to  correctly  classify 

□  Exceptions:  National  Sensors  and  other  programs 

>  Fix  -  ability  of  sensor  system  to  localize  TOIs 

□  Function  of  sensor’s  collective  errors  in  range  and  bearing 

>  Track  -  ability  of  sensor  system  to  maintain  contact  and 
establish  course  and  speed  information 

>  Target  -  ability  of  sensor-weapon  system  pair  to  achieve 
targeting  quality  solution 

>  Engage  -  sensor-weapon  system  ability  to  consummate 
the  kill  chain 

>  Assess  -  ability  of  sensor-weapon  system  pair  to 
ascertain  engagement  success 

□  May  be  some  staff  functions  in  this  area 


Generally  Broader  in  Scope ...  Than  pure  experimentation 


REM  Program 

What  does  it  Provide? 

Surface  Warfare  Center  of  Excellence  _ 


>  Acquisition  Community 


□  Provides  legacy  system  performance  feedback  -  identifies  potential 
upgrade  points 

□  Eariy  evaluation  of  developmental  systems 

□  Identifies  human  factor  issues 

□  Operational  capability  vs.  engineering  assessment 


>  Research  &  Deveiopment  Community 

□  Focuses  R&D  efforts  in  capability  gap  areas 

□  Venue  for  evaluating  developmental  efforts 

>  Training  Community 

□  At-Sea  Operational  Feedback  loop 

□  Data  for  FST-U  implementation 

□  Identifies  NCEA  requirements 


Real  World  -  Operational  Environment  -  Representative  Threats 

Systems  Tactically  Employed  -  ROC  &  POE  Validation  -  Fleet  Operators 


REM  Program 

What  does  it  Provide? 

Surface  Warfare  Center  of  Excellence  _ 


>  Warfighter 


□  Hot  Wash-up  Rapid  feedback  on  exercise  results  -  90%  solution 

□  What  happened  -  may  not  have  answers  to  why  it  happened 

□  Current  capability  assessment  in  areas  of  immediate  interest 

□  Feedback  for  MCO  planning 

>  Doctrine  &  Tactics  Community 


□  Tactical  development,  validation,  verification  &  refinement  venue 

□  Identifies  and  develops  non-materiel  solutions  to  mitigate  or  resolve 
current  capability  gaps 


Reai  Worid  -  Operationai  Environment  -  Representative  Threats 
Tactically  Focused  -  Areas  of  Interest  -  What  it  takes  to  win... 


REM  Program 

What  makes  It  Effective? 

Surface  Warfare  Center  of  Excellence  -  - = 


>  Annual  Goal  End  State  established 


>  Objectives  agreed  upon  up  &  down  Chain  of  Command 

□  Prioritized  to  achieve  End  State  Goais 

>  Analysis  Plan  developed  that  supports  objectives 

>  Exercise/Experimentation  Time  &  Assets  Allocation 

■  Determined  by  assessing  the  objective  data  coiiection  requirements 

>  Post-ex  Analysis  -  Levels  of  Granularity 

□  Participant  Feedback 

□  Determine  progress  towards  end-state  goais 

□  Assess  key  performance  parameters 

□  Update  trend  anaiysis 

>  Review,  Revise  &  Refine  Annual  Plan  to  attain  Goals 

□  Compare  to  hypotheses  -  make  corrections  as  necessary 

□  Feedback  to  Sponsor  &  Stakehoiders 
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SHAREM  History 

Surface  Warfare  Center  of  Excellence  _ 


>  Established  by  CNO  in  1969  to  quantitatively  assess 
surface  ship  ASW performance  (circa  UPTiDE 
Ex  peri  men  ta  tion) 


□  COMSURFWARDEVGRU:  Executive  Agent 


>  Assess  surface  ship  and  ASW  force  performance 

>  Identify  fleet  deficiencies  and  shortfalls 

>  Informs  Fleet  investment  Strategy  -  identifying 
requirements  and  guiding  procurement  process 


>  Provide  test  bed  to  evaluate  new  technologies,  systems, 
and  prototypes 

>  Maintain  database  on  ASW  sensor  &  weapon  performance 

>  Site  specific  areas  of  interest  shift  in  1992  -  more 
tactically  relevant/representative  -  response  to  C5F 
request 


Ships’  ASW  Readiness,  Effectiveness  Measuring  (SHAREM)  Program 


SHAREM  Evolution 


Surface  Warfare  Center  of  Excellence 


>  Originally  purely  an  engineering  evaluation 


□  /ASH/  systems  were  groomed 

□  Performance  evaluated  against  established  acceptance  metrics 

>  Gradual  shift  to  tactical  evaluation 


□  Exercises  limited,  primarily,  to  East  Coast  assets 

□  Typically  regarded  as  goal-post  to  goal-post  events 

>  Area  of  interest  shift  in  early  1990s 

□  Conduct  exercises  in  forward  deployed,  tactically  significant 
areas 

□  Participants  on  deployment 

□  Representative  threats  and/or  threat  tactics 

□  Assess  current  state  expected  performance 

□  Combination  structured  and  freeplay  evaluations 


13 


Think  Again , 


Surface  Warfare  Center  of  Excellence 
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SHAREM 

3  7  Years  of  Lessons  Learned 


Surface  Warfare  Center  of  Excellence 


'Y  Background/Overview 
❖  Improved-SHAREM  Process 

□  Metrics 

□  Lessons  Learned 
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Surface  Warfare  Center  of  Excellence 


Improved  SHAREM 

Analysis  Thread 

rpi  - - 


Objectives 

Standard 

Detection 
-Classification 
-Localization 
-Attack 
Specific 
FLT  CDRs 
DESRONs 
System  Tests 


COMEX 
(-6  mos) 


COMEX 
(-5  mos) 


Analysis  Plan 

-Define  COI 
-Analytical 
Procedure 
-Evaluation 
Criteria 
-Specifies 
data  to  be 
collected 


Process  Management  Plan 

Provides  processes/procedures 
for  SHAREM  Program  execution 


Test  Plan 

-Detailed 
Procedures  for 
Conducting 
Events 

Data  Collection 

Plan 

Data  collected 
supports 
objectives 
and  R&A 


Conduct  Exercise 
(ADCE/Observers) 


BaclTceir^ 

Provide  in-stride  R&A  and 
Event  Replay  (AVI’s) 


-Hot  Wash  Brief 
-Quicklook  MSG 
-Flag  Brief 
-R&A  Report 


Improved  SHAREM  Timeline 


COMEX 
(-4  mos) 


COMEX 
(-3  mos) 


COMEX 
(-2  mos) 


COMEX 
(-1  mos) 


Conduct 

Exercise 


FINEX 
(+1  mos) 


FINEX 
(+2  mos) 


FINEX 
(+3-6  mos) 


IPC 

MPC 

Analysis  Plan  i 

FPC 

A  A 

Hot  Wash  E 

1 

T 


R&A  Report 


Brief 


Flag  Brief  (45  days) 


Test  Plan/Data  Collection  Plan 
(6  weeks  prior  to  COMEX) 


Reach  Back  Cell  Stands  Up 


Quicklook  MSG  (7  working  days) 
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Surface  Warfare  Center  of  Excellence 


I-SHAREM 

Program  Specific  Goals 


>  Provide  a  tactical  development  and  validation  vehicle 

□  Realistic,  tactically  significant  environments 

□  Multi  -  platform  USW  force:  ships,  aircraft,  submarines,  SURTASS 

□  Coalition  partner  involvement 

□  Ground  truth  track  reconstruction 

□  Extensive  analysis  &  meteorological  impact  assessment 

□  Validated  results  for  training,  doctrine,  and  programmatic  decisions 

□  Identify  and  provide  non-materiel  solutions 

□  Mitigate  current  gaps  and  optimize  fielded  systems  with  TTP 

□  Address  Warfighter  concerns  and  provide  rapid  feedback 


>  Sensor  &  Weapon  System  performance  assessment 

□  Employed  in  tactical  areas  of  interest 

□  Representative  threats  &  tactics  (or  emulations  thereof) 

□  Performance  across  the  ‘kill  chain’ 

□  Identify  materiel  performance  shortfalls 
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Surface  Warfare  Center  of  Excellence 


I-SHAREM 

Measurements  &  Evaluations 


>  Acoustic  &  non-acoustic  sensor  detection  & 
ciassification  effectiveness 

>  ASW  iocaiization  procedures  &  tactics;  accuracy  & 
timeiiness 

>  ASW  attack  procedures,  weapons  &  tactics 

>  ASW  C4i  and  data  fusion  in  Task  Force  operations 

>  Acoustic  and  non-acoustic  vuinerabiiity  to  detection 
and  attack  by  (diesei)  submarines 

>  Abiiity  of  ASW  forces  to  expioit  the  environment 
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I-SHAREM 

Hot-  Wash  Products 


Surface  Warfare  Center  of  Excellence 


4 
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Multi-Static  ASW 
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Surface  Warfare  Center  of  Excellence 

22  nm 


Multi-Static  ASW 

Tactical  Experiment 


30  nm 


□ 

1 

O) 

■ 

o 

□ 

0.8-0.9 

□ 

0.7-0.8 

□ 

0.6-0.7 

□ 

0.5-0.6 
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SHAREM 

3  7  Years  of  Lessons  Learned 

Surface  Warfare  Center  of  Excellence  -  - 

Background/Overview 
'Y  Improved-SHAREM  Process 
❖  Metrics 

□  Lessons  Learned 
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Improved  SHAREM  Analysis 

Surface  Warfare  Center  of  Excellence  _  _  _  - 


Best  metrics  are  independent  of  platform  actions  or  exercise  design 


Probability  of  Detection  as  a  Function 

of  Laterai  Range 

Proper  implementation  requires  imposing 
course  and  speed  restrictions  on  the  target. 

Hoid  Time 

Assumes  the  presence  of  a  target  in 
the  area.  Difficult  to  normalize  size  and 
shape  of  the  target  OPAREA. 

Contact  Validity  Ratio 

Assumes  the  presence  of  a  submarine  in 
the  operating  area  and  is  a  function  of 
target  operating  area  and  target  density. 


Cumuiatiye  Probability  of  Detection 

Accounts  for  target  distance  regardless  of 
target  movement. 

Time  Expended  Tracking  Inyalid  Contacts 

Independent  of  submarine  location  or  size  of  the 
target  operating  area  “cost  of  doing  business” 
concern  relatively  independent  of  other  factors. 

Raise  Contact  Rate 

Considers  invalid  contacts  only.  Submarine 
location  and  OPAREA  size  do  not  matter. 


Improved  SHAREM  Metrics 


“An  exercise  without  analysis  is,  at  best,  a  demonstration...” 


Find 


Improved  SHAREM  Metrics  ^ 

- ^ 


Surface  Warfare  Center  of  Excellence 


Track 


Tracking 

Percent  Time  Tracking  Invalid  Contacts 
Ability  to  maintain  track 


ngage 


Attack  Effectiveness 

Invalid  Attack  Rate 
Invalid  Attack  Probability 
Valid  Attack  Probability 


T 


Assess 


Independent  &  Defensible  Measures 


Whafs  Reported  -  Vice  Quoted 

Presentation  Matters 


Surface  Warfare  Center  of  Excellence 


Event 

False 

Alarm 

False 

Alarm 

per 

Day 

Exp 

Det 

Det/ 

Exp 

(%) 

Exp 

Time 

Hold 

Time 

f HoldK 
j  Exp  \ 

/  \ 

09100 

1 

3 

84 

47 

56 

3:59:30 

3:25:41  ; 

I  86  \ 

10400 

0 

0 

2 

2 

100 

4:23:00 

3:02:56 

70 

10900 

1 

2 

7 

7 

32 

2 

12 

1 

38 

50 

6:26:47 

1:02:00 

5:02:18 

0:26:21 

78 

42 

12100 

0 

0 

2 

2 

100 

2:23:00 

1:55:11 

81 

12500 

0 

0 

11 

9 

82 

8:50:55 

8:44:40 

99 

12800 

0 

0 

N/A 

N/A 

N/A 

N/A 

N/A 

N/A 

14100 

3 

4 

14 

7 

50 

1:09:48 

1:10:16 

100 

15100 

3 

2.1 

29 

9 

31 

00:49:11 

0:09:05 

19 

Struc 

1 

0.89 

99 

60 

61 

19:36:25 

17:08:28 

87 

F-P 

13 

4.1 

77 

Qi2> 

C^38  ^ 

9:26:59 

6:48:00 

\  72  1 

Total 

14 

r  y 

2.8 

r  y 

176 

f  y 

89 

r  y 

50 

f  \ 

29:03:24 

r  y 

23:56:28 

r  ^ 

^  \  / 
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I-SHAREM 


One  Exercise  —  One  Day 

Surface  Warfare  Center  of  Excellence  -  - = 
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I-SHAREM 

Same  Exercise  -  Another  Day 


Surface  Warfare  Center  of  Excellence 


Judging  overall  performance  based  on  data  from  one  event  =  Danger! 


SHAREM 


3  7  Years  of  Lessons  Learned 


Surface  Warfare  Center  of  Excellence 


'Y  Background/Overview 
'Y  Improved-SHAREM  Process 
^  MOEs&MOPs 
*>  Lessons  Learned 
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Lessons  Learned 

Evolving  to  Match  Environment 


Surface  Warfare 


Center  of  Excellence 


>  Exercises  designed  to  objectives  -  Sequence  is  criticai 

□  Start  with  objectives  -  iimit  the  number  or  acknowiedge  faiiure 

□  Deveiop  anaiysis  pian  to  support  objectives 

□  identify  data  requirements  based  on  objectives  and  anaiysis  pian 

□  Design  exercise  (experiment)  to  support  objectives 

□  Detaiied  data  coliection  and  anaiysis  pian  up-front! 

>  Expanded  use  of  Limited  Objective  Experiments  (LOEs) 

□  Smaiter  scaie  easier  to  manage, 

□  Better  ensures  goat  attainment 

>  Trade-offs  between  exercise  and  experiment  requirements 

□  Requires  eariy  “buy-in”  on  priorities  by  aii  concerned 

□  Prioritization  of  events  to  ensure  reaiization  of  objectives 

□  incorporation  of  desires  requires  deconfiiction 

>  SHAREM IPT  sets  objectives  and  scheduies  venues 

SWDG  remains  th£  honest  broker  for  Surface  Warfare  Capabilities 


Lessons  Learned 

Technologies  -  Only  part  of  the  solution 


Surface  Warfare  Center  of  Excellence 


>  Unrealized  potential  remains  when  systems  are  used 
without  complete  understanding 


>  New  systems  require  different  thinking  &  TTP 

□  Systems  lacking  TTP  will  be  sub-optimized 

□  New  systems  may  not  perform  as  advertised  -  but  will  definitely  be 
sub-optimized  without  accompanying  TTP 


>  Technology  inserts  are  not  silver  bullets  and  may  only 
have  niche  applications 

□  Systems  need  to  be  integrated  &  interoperable 

>  Improved  SHAREM  analysis  provides  comparable 
answers  between  disparate  technologies 


**Any  order  of  magnitude  change  requires  completely  new  thinking  -  old  methods 

may  no  longer  be  applicable...  -  Richard  Hamming 


Surface  Warfare  Center  of  Excellence 


Lessons  Learned 

Training  &  Proficiency 


>  Training  provides  basics  -  practice  yields  proficiency 

>  Establishing  training  &  proficiency  levels  key 

□  Assesses  tralning/proficlency  Impact  on  overall  performance 

□  Informs  training  community  on  knowledge  gaps 

□  Establishes  ‘remedial’  and/or  ‘refresh’  training  requirements 

□  Guides  synthetic  trainer  developments 

□  Informs  Fleet  Response  Plan  requirements 

>  Feedback  for  acquisition  and  research  &  development 
communities 

>  Consistent  &  coherent  metrics  needed 
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Lessons  Learned 


Surface  Warfare  Center  of  Excellence 


Metrics 


>  Metrics  ‘musts’ : 


□  Translatable 

□  Defensible 

□  Comparable 

□  Understandable 

□  Reproducible 

□  Objective  based 

□  Identify  KPPs 

□  Pass  Independence  test 

>  Introduction  of  new  metrics  will  meet  resistance 

>  Single  point  evaluations  are  suspect 


"Not  everything  that  counts  can  be  counted,  and  not  everything  that  can  be 
_ counted  counts. "  -  Einstein  (attributed) _ 
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Lessons  Learned 

Kitchen  sink... 


>  Quicklooks  are  always  inaccurate 

>  Human  factors  and  memories  can  skew  results 

>  Piggy-backed  evolutions  will  generally  fall  short 

>  True  Freeplays  yield  little  insight  -  often  result  in  “no 


tests” 


□  Freeplays  with  objective  focused  instructions  can  produce  insight 

>  All  results  scrutinized  -  especially  those  contrary  to 
perceptions 

>  Repeat  Lessons  Learned  are  Ongoing  Discrepancies! 

>  Rapid  warfighter/participant  feedback  criticai 

□  Best  practices  may  only  apply  to  given  circumstances 

□  Documentation  is  key  to  moving  forward  and  not  forgetting. . . 
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I-SHAREM  ->  SURFREM 

Extension  Across  All  Warfare  Areas 
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SURFREM  ...The  Future  Of  The  SHAREM/MIREM  Process 


OPNAV 

NWDC 

CFFC 

PEO 

PA  Surface  Force 

ONR  (FNC) 

Theater 

OPTEVFOR 

OA  Sea  Shield 

Joint  (ACTD) 

OA  Sea  Strike 

SHAREM 

MIREM 

ADREM 

SUWREM 

EWREM 

GWQIBEM 


Executive  Steering 
Group(s) 

Fleet  Collaborative 
Team  (Surface) 

Sea  Triai/STIMS 


1 


FLEET  LiAISON 

(C2F/C3F/C5F 
C6F/C7F) 
Scheduling 
Conference 


Exercise  Planning  Process  (IPC/MPC/FPC) 

CSG,  ESG,  DESRON,  PHIBRON,  Ships,  CSFTL/P,  TTGL/P  &  ATGL/P 


Exercise 


Immediate 

Feedback 


10-Day 

Quickiook 


OPS 

Research 


OPS 

Anaiysis 


Surface  Force  Programmatic  Input...  Tactical  Development...  Analytical  Feedback 


